Neural network with unbounded activation functions is universal approximator
نویسندگان
چکیده
منابع مشابه
Neural Network with Unbounded Activations is Universal Approximator
Abstract This paper presents an investigation of the approximation property of neural networks with unbounded activation functions, such as the rectified linear unit (ReLU), which is the new de-facto standard of deep learning. The ReLU network can be analyzed by the ridgelet transform with respect to Lizorkin distributions. By showing three reconstruction formulas by using the Fourier slice the...
متن کاملOn the Use of Neural Network as a Universal Approximator
Neural network process modelling needs the use of experimental design and studies. A new neural network constructive algorithm is proposed. Moreover, the paper deals with the influence of the parameters of radial basis function neural networks and multilayer perceptrons network in process modelling. Particularly, it is shown that the neural modelling, depending on learning approach, cannot be a...
متن کاملUniversal Approximator Property of the Space of Hyperbolic Tangent Functions
In this paper, first the space of hyperbolic tangent functions is introduced and then the universal approximator property of this space is proved. In fact, by using this space, any nonlinear continuous function can be uniformly approximated with any degree of accuracy. Also, as an application, this space of functions is utilized to design feedback control for a nonlinear dynamical system.
متن کاملUniversal Approximation with Non-sigmoid Hidden Layer Activation Functions by Using Artificial Neural Network Modeling
Neural networks are modeled on the way the human brain. They are capable of learning and can automatically recognize by skillfully training and design complex relationships and hidden dependencies based on historical example patterns and use this information for forecasting. The main difference, and at the same time is biggest advantage of the model of neural networks over statistical technique...
متن کاملThe Learning Dynamcis of a Universal Approximator
The learning properties of a universal approximator, a normalized committee machine with adjustable biases, are studied for on-line back-propagation learning. Within a statistical mechanics framework, numerical studies show that this model has features which do not exist in previously studied two-layer network models without adjustable biases, e.g., attractive suboptimal symmetric phases even f...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Applied and Computational Harmonic Analysis
سال: 2017
ISSN: 1063-5203
DOI: 10.1016/j.acha.2015.12.005